Minimizing Mistakes and Embracing Uncertainty

August 2005 | Volume 2 | Issue 8 | e272 " Truth in science can be defi ned as the working hypothesis best suited to open the way to the next better one. " —Konrad Lorenz, Austria S cientifi c truth is a moving target. In the process of peer review, authors, reviewers, and editors work together to minimize the reporting of false results. However, even if one assumes no bias, wrongdoing, or ignorance on the part of any of the individuals involved— which is unrealistic, no doubt—chances are that some fi ndings will turn out to be false. But is it inevitable, as John Ioannidis argues in an Essay in this issue of PLoS Medicine (DOI: 10.1371/ journal.pmed.0020124), that the majority of fi ndings are actually false? Although his calculations are based on assumptions about complex scenarios that we do not fully understand—as is true for most research projects—Ioannidis argues convincingly that many published fi ndings will turn out to be false. Ioannidis is not the fi rst to raise some of these concerns. Indeed, there are already initiatives under way that seek to address them. Increasingly, researchers design individual studies, systematic reviews, and meta-analyses using Bayesian statistics, in which the issue of pre-study odds is taken into account. And issues such as reducing sources of bias when assessing evidence are addressed in the methodology used by the Cochrane Collaboration in the production of its systematic reviews. Ioannidis doesn't defi ne " fi ndings, " but it seems important to attempt to separate data (" in this study 5% of people examined who lived in San Francisco from 1965–1970 developed lung cancer compared with 20% of people studied who lived in Anchorage ") from conclusions (" lung cancer rates are higher in Anchorage than San Francisco ") and hypotheses (" cold weather exacerbates the consequences of smoking "). Hypotheses will inevitably change, as they depend not only on the study but also on the context of other relevant research and knowledge. Conclusions are also often based on current knowledge and assumptions, and, thus, subject to change. The data should be more robust; for instance, other researchers applying the same methods to study the same group of patients at the same time should be able to generate the same data. However, research progress depends on conclusions being tested elsewhere. The major issue about the truth of research fi ndings would …